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Abstract 

Background: Anesthesia information management system (AIMS) records should be designed and configured to 
facilitate the accurate and prompt recording of multiple drugs administered coincidentally or in rapid succession. 

Methods: We proposed two touch-screen display formats for use with our department's new EPIC touch-screen 
AIMS. In one format, medication "buttons" were arranged in alphabetical order (i.e. A-C, D-H etc.). In the other, 
buttons were arranged in categories (Common, Fluids, Cardiovascular, Coagulation etc.). Both formats were 
modeled on an iPad screen to resemble the AIMS interface. Anesthesia residents, anesthesiologists, and Certified 
Registered Nurse Anesthetists (n = 60) were then asked to find and touch the correct buttons for a series of 
medications whose names were displayed to the side of the entry screen. The number of entries made within 
2 minutes was recorded. This was done 3 times for each format, with the 1 st format chosen randomly. Data were 
analyzed from the third trials with each format to minimize differences in learning. 

Results: The categorical format had a mean of 5.6 more drugs entered using the categorical method in two 
minutes than the alphabetical format (95% confidence interval [CI] 4.5 to 6.8, P < 0.0001). The findings were the 
same regardless of the order of testing (i.e. alphabetical-categorical vs. categorical - alphabetical) and participants' 
years of clinical experience. Most anesthesia providers made no (0) errors for most trials (N = 96/1 20 trials, 
lower 95% limit 73%, P < 0.0001). There was no difference in error rates between the two formats (P = 0.53). 

Conclusions: The use of touch-screen user interfaces in healthcare is increasingly common. Arrangement of drugs 
names in a categorical display format in the medication order-entry touch screen of an AIMS can result in faster 
data entry compared to an alphabetical arrangement of drugs. Results of this quality improvement project were 
used in our department's design of our final intraoperative electronic anesthesia record. This testing approach using 
cognitive and usability engineering methods can be used to objectively design and evaluate many aspects of the 
clinician-computer interaction in electronic health records. 
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Background 

An Anesthesia Information Management System (AIMS) 
has two components: (1) automatically validated data 
from the anesthesia machine and the physiological moni- 
tor and (2) the data manually entered by the anesthesia 
provider including events, medications, information on 
airway management, etc. Anesthesia residents, anesthe- 
siologists, and Certified Registered Nurse Anesthetists 
(anesthesia providers) using AIMS need to record the 
manual data while simultaneously engaged in other 
more vital tasks. One example involves medications: 
an anesthesia provider may make dozens of drug-dose 
entries within a few hours. Typically, no pharmacist or 
nurse is involved in medication delivery. The anesthesia 
provider decides on the drug, draws it up, and adminis- 
ters it, often within seconds or minutes. The rate of 
required drug data-entry is not uniform over time. Many 
more entries are required during the relatively short 
period of induction and other critical phases than during 
maintenance. A well-designed AIMS should facilitate the 
accurate and rapid recording of multiple drugs. 

Our hospital made the decision to implement EPICs 
AIMS (Epic Systems, Verona, WI). This system allows 
the user to configure many of its components - including 
drug entry formats. During our development period, we 
performed a systematic search for experimental and ob- 
servational studies of the impact of display format on the 
rate of entry of medications (and on errors) and found 
none for any specialty. a We therefore undertook a study 
to determine which of two basic entry formats was better 
for fast and accurate data entry. We evaluated how de- 
sign variation (alphabetical versus categorical) and user 
variation (users with different levels of clinical experi- 
ence) affects the task and the occurrence of errors. 

Methods 

This project was performed as a quality improvement 
project. No records were maintained of the names of 
any participants. Results were used for the design of our 
final AIMS order entry screen. The University of Iowa 
Institutional Review Board reviewed our application for 
publication of the results and concluded "this is not 
human subjects research." 

Two touch-screen display formats were programmed 
on an iPad (Apple Inc., Cupertino, CA) to resemble the 
proposed AIMS medication order entry touch screens. 
The iPad program was Web-based, utilizing ASP.net, 
jQuery and SQL Server. Each time a participant selected 
a drug, the server recorded the result of that selection. 
Each trial lasted 2 minutes. The number of entries com- 
pleted within the 2 minutes was recorded, along with 
any entry errors. 

The two formats were alphabetical and categorical. For 
the first, the entry screen was constructed with "tabs" 



across the top labeled A-C, D-H, I-O and P-Z. Under 
each tab, individual medication "buttons" were listed 
in alphabetical order (e.g. acetaminophen, albuterol, 
alfentanil, atropine etc.). There were roughly 25-30 
medications under each alphabetical tab. For categor- 
ical format, the entry screen was constructed with tabs 
across the top for Common Drugs, Fluids and Electro- 
lytes, Cardiovascular, Coagulation, Regional Anesthetic, 
Antibiotics and Other Drugs. Under each category, drugs 
were grouped by category (e.g. all neuromuscular block- 
ing drugs together, all opioids together etc.) Drugs listed 
under Common Drugs were chosen based on an exten- 
sive review of anesthetic records, and represented the 
29 medications used most commonly in our operating 
rooms. Colors were used to label medications to match 
American Society for Testing and Materials (ASTM) 
standards as much as possible. Screen shots of example 
screens for both formats are shown in Figures 1 and 2. 

There were 132 medications included in each of the 
study formats. 

The study was conducted in the operating room envir- 
onment but not during periods of patient care. Three 
groups of participants were chosen, based on their years 
of clinical anesthesia experience: anesthesia residents 
and student registered nurse anesthetists (SRNA) with 
<1 year of clinical anesthesia experience; anesthesia resi- 
dents, fellows, and SRNA with 1 to 3 years of anesthesia 
experience; and anesthesiologists and Certified Regis- 
tered Nurse Anesthetists, all with > 4 years of experi- 
ence. The convenience samples were N = 20 participants 
per group, selected by the first author. Everyone invited 
to participate chose to do so. Each participant partici- 
pated in the study individually, not in groups. 

Testing protocol 

Participants were handed the iPad, which displayed a 
brief introduction of the study and confirmed their will- 
ingness to participate in the study. Each participant 
entered his or her years of clinical experience. Successive 
participants in each group were assigned in alternating 
order to an initial format, either alphabetical or categor- 
ical. Next, the participants underwent a training exercise 
to familiarize them with the system. The participants 
were presented with the names of countries, grouped 
either alphabetically or categorically by continent based 
on the initial assignment. The participant was prompted 
with the name of a country in the lower right corner of 
the screen and was then required to locate that country 
under one of the tabs, and tap the button for that coun- 
try. If correct, the country moved automatically to the 
left side of the screen in a vertical list. A different country 
name then replaced the current country in the lower 
right portion of the screen. The demonstration ended 
when the participant successfully tapped four countries. 
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Midazolam 



Fentanyl 



Lidocaine 



Propofol 



A-C D-H 1-0 



Penicillin GK 


Ropivacaine 0.5% 


Vecuronium 


Phentolamine 


Ropivacaine 0.75% 


Verapamil 


Phenylephrine 


Sodium 
Bicarbonate 4.2% 




Procainamide 


Sodium 
Bicarbonate 7.5% 


Propofol 




Protamine 


Tetracaine 


Remifentanil 


Thiopental 




Tranexamic Acid 


Ropivacaine 0.2% 


Vancomycin 



Phenylephrine 



Figure 1 Screen shot of alphabetical display format. This drug entry screen was constructed with "tabs" across the top labeled A-C, D-H, 1-0 
and P-Z. Under each tab, individual drugs were listed in alphabetical order. 



At the end of this training period, the participant 
tapped the Start Trial button. Based on the assignment 
either the alphabetical or categorical format appeared. 
The name of a drug appeared on the lower right corner 
of the screen and the participant was required to find 
that drug under the various tabs and touch the correct 
button to complete its entry. When the correct drug was 
selected, a new drug name would appear on the screen. 



Participants had 2 minutes to find and enter as many 
drugs as possible from a list of 25 drugs that were dis- 
played in an order (Table 1). Should they complete the 
25 drugs within the allotted time, the list would cycle 
back to the first drug. At the end of the 2 minutes trial, 
they were shown their results (number of drug entered, 
number of errors) and directed to begin the next trial. 
The second trial used the same format and requested 



Fentanyl 



Phenylephrine 



Lactated Ringers 



Fluids & Electrolytes Cardiovascular Coagulation Regional Other Antibiotics 



Midazolam 



Midazolam PO 



Propofol 



Etomidate 



Thiopental 



Phenylephrine 



Ephedrine 



Alfentanil 



Fentanyl 



Hydromorphone 



Morphine 



Meperidine 



Acetaminophen - 
PR 



Droperidoi 



Cisatracurium 



Neostigmine 



Lactated Ringers 




Dexamethasone 



Lidocaine 



Figure 2 Screen shot of categorical display format. The drug entry screen was constructed with "tabs" across the top for Common Drugs, 
Fluids and Electrolytes, Cardiovascular, Coagulation, Regional Anesthetic, Antibiotics and Other Drugs. Under each category, drugs were grouped 
by category. 
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Table 1 Order of Drugs in Trial 1, Trial 2 and Trial 3 



Trial 1 


Trial 2 


Trial 3 


Midazolam 


Fentanyl 


Lidocaine 


Fentanyl 


Phenylephrine 


Cefazolin 


Lidocaine 


Lactated Ringers 


Bupivacaine 0.25% 


Propofol 


Lidocaine 


Midazolam 


Succinylcholine 


Epinephrine 


Heparin 


Phenylephrine 


Propofol 


Propofol 


Rocuronium 


Succinylcholine 


Lactated Ringers 


Ephedrine 


Ephedrine 


Succinylcholine 


Lactated Ringers 


Rocuronium 


Albumin 5% 


Dexamethasone 


Cefazolin 


Rocuronium 


Fentanyl 


Rocuronium 


Fentanyl 


Cefazolin 


Albumin 5% 


Insulin 


Albumin 5% 


Insulin 


Fentanyl 


Epinephrine 


Dexamethasone 


Epinephrine 


Rocuronium 


Fentanyl 


Dexamethasone 


Norepinephrine 


Norepinephrine 


Norepinephrine 


Insulin 


Midazolam 


Ondansetron 


Heparin 


Heparin 


Morphine 


Ketorolac 


Bupivacaine 0.25% 


Ketorolac 


Morphine 


Glycopyrollate 


Phenylephrine 


Lactated Ringers 


Lactated Ringers 


Lactated Ringers 


Ondansetron 


Neostigmine 


Rocuronium 


Neostigmine 


Morphine 


Fentanyl 


Glycopyrollate 


Ondansetron 


Neostigmine 


Bupivacaine 0.25% 


Ketorolac 


Glycopyrollate 



The lists of 25 drugs were created from the 22 commonly used anesthestic 
drugs in our department. The list repeated from the top if/when a clinician 
had entered all 25 drugs within 2 minutes and had time remaining. Because 
the list contains 25 drugs and yet 22 drugs were used often, there are 
duplicates within the lists: Fentanyl, Lactated Ringers, and Rocuronium. 



the entry of the same drugs as the first trial; however, 
the order with which drugs were listed was changed. 
This same process was repeated for a third trial Partici- 
pants were next instructed that the format would change 
from the previously viewed alphabetical format to a 
categorical format, or vice-versa. They repeated the 
demonstration with countries using the new format. 
They then completed an additional three 2 minute long 
drug-entry trials using the new format. 

The drugs were displayed in the same sequence for 
corresponding trials between the different formats 
(i.e. the sequence of drugs was the same for Trial 4 as 
it was for Trial 1). The total time for each participant 
was approximately 20 minutes. 

Analysis 

To avoid any problems with a "learning curve," only the 
number of drugs entered on the third trial with each 



format was used in our primary analysis. Numbers of 
drugs entered in each 2 minute period were compared 
pairwise by participant using two-sided one-group Stu- 
dents t tests. The pairwise differences were reported as 
mean ± standard deviation. Analyses were repeated after 
stratifying by the initial format viewed, alphabetical or 
categorical. Those differences were consistent with nor- 
mal distributions (Shapiro- Wilk P > 0.05). 

Statistical analyses of the numbers of medications 
entered incorrectly were performed using two-group 
permutation tests. This test is like the Wilcoxon signed- 
ranks test, but includes the 0s. The zeros mattered be- 
cause most participants had no errors for most trials. 
Confidence intervals for median differences were calcu- 
lated using the Hodges-Lehman method. All results were 
calculated using exact methods (StatXact-9, Cytel Soft- 
ware Corporation, Cambridge, MA). 

Results 

Providers using the categorical format had a mean of 
32.5 drugs entered in two minutes in the third trial com- 
pared to 26.8 drugs in the alphabetical format. The mean 
pairwise difference was 5.6 ± 4.4 more drugs entered 
using categorical than alphabetical format (95% confi- 
dence interval 4.5 to 6.8, P < 0.0001). The order of test- 
ing did not alter this result. Individuals tested first with 
the alphabetical format averaged 6.0 ±4.8 more drugs 
entered by category (95% CI 4.2 to 7.7, P < 0.0001) 
while those tested first with the categorical format aver- 
aged 5.3 ±4.0 more drugs entered via the categorical 
format (95% CI 3.8 to 6.8, p< 0.0001). A similar pat- 
tern was observed when providers were analyzed by years 
of experience. 

Since categories rely on learning, a learning effect was 
expected and observed. For the first trial, alphabetical was 
similar to categorical, providing 0.9 ± 5.5 more drugs 
entered than categorical (95% CI -0.6 to 2.3, P = 0.23). By 
the third trial, the categorical format resulted in 5.6 ± 4.4 
more drugs entered in two minutes (Figure 3). 

Most anesthesia providers made no (0) errors for most 
trials (N = 96/120 trials, lower 95% limit 73%, P < 0.0001). 
There was no difference in error rates between alpha- 
betical and categorical templates at both the first 
(P = 0.54) and third trials (P = 0.53). The corresponding 
95% confidence intervals for the median differences were 
0 to 0 and 0 to 1 (categorical less), respectively. 

Discussion 

The designing and layout of the user interface of an elec- 
tronic medical record can influence task load, time to 
task completion, and number of errors of cognition 
associated with the identification, and subsequent use, of 
relevant patient data by the clinical provider [1]. Among 
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Trial 1 Trial 2 Trial 3 

Trial Count 

Figure 3 Bar Chart showing the average number of drugs entered in the alphabetical and categorical formats during Trial 1, Trial 2 
and Trial 3. The x-axis shows the trial number and the y-axis shows the number of drugs entered in each trial. Only the number of drugs 
entered in Trial 3 was used in the primary analysis to avoid any problems with "learning curve". The magnitude of the learning curve was 
estimated by taking for each participant the average of two pairs of numbers of drugs entered: (Trial 3 categorical - Trial 1 categorical) and 
(Trial 3 alphabetical - Trial 1 alphabetical). The meant standard deviation among participants of the average differences were 8.1 ±3.4 drugs 
entered (P < 0.0001 in comparison to 0; N = 60). 

V J 



US anesthesiologists using AIMS, 40% surveyed identi- 
fied less efficient workflow as a problem with AIMS [2]. 
With a paper record, writing down the name of a drug 
and dose administered takes a couple of seconds. With a 
touch screen interface, the anesthesia provider has to 
find the drug from the list of available (e.g., 132) medica- 
tions and document it. 



Cognitive demands introduced by the system on physi- 
cians who are already engaged in many other vital tasks 
contribute to decrease in efficiency [3]. Horsky and col- 
leagues developed a methodology for the characterization 
of cognitive demands of a medical information system, 
which was based on the distributed resources model, an 
approach that describes the dimensions of user interfaces 



02 (L/min) 



Midazolam (mg) 



Fentanyl (meg) 



Active Common 



Fluids. 
Electrolytes 



Cardiovascular 


Coagulation 


Regional 
Meds 


Miscellaneous 
Meds 


IM 
Drugs 


Topicals 



Lidocaine 


Fentanyl 


Lactate d Ringers 


Midazolam 


Hydromorphone 










| Midazolam PO I 




Vecuronium 








Propofol 


Meperidine 


111111 




Etomidate 


Remifentanil 


Atropine 




Ketamine 


'///////////// 
Naloxone 


Glycopyrrolate 




Acetaminophen - 
PR 
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Ondansetron 




| Succinylcholine | 


Droperidol 



Close 



Figure 4 Final Anesthesia medication order entry screen that was implemented in the anesthesia information management system 
based on the study. This figure can be compared to the experimental Figure 2. 
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that introduce unnecessary cognitive complexity. Seman- 
tic matching rather than alphanumeric ordering or strict 
hierarchies may expedite searches for orders, sets, and 
text-based values in pick lists that frequently contain 
dozens of items [4]. 

Human-computer interaction has been an important 
part of cognitive science for decades [5]. Usability of a 
computer system is defined as the capacity of the system 
to allow users to carry out their tasks safely, effectively 
and efficiently [6]. Usability testing refers to the evalu- 
ation of information systems that involves testing of 
participants who are representative of the target popu- 
lation [7]. Our study resulted in usability testing of the 
medication order entry screen of touch screen AIMS by 
our anesthesia providers. Cognitive task analysis (CTA) 
for evaluation of medical systems represents the integra- 
tion of work from the field of systems engineering and 
cognitive research in medicine [7]. In our study, CTA 
involved the effect of design variation and user variation 
in the completion of task and error rate in an AIMS. 

Our experimental study had multiple limitations. First, 
this project was performed as a quality improvement 
project, using a convenience sample of available clini- 
cians, with no formal power analysis performed until 
there were N = 60 participants at which time the project 
was stopped based on the findings. Second, the 22 medi- 
cations studied are those commonly used in our depart- 
ment (Table 1). Results may be different for departments 
routinely using fewer or greater numbers of medications. 
Third, we did not include interruptions or distractions. 
Although we do not expect that results would be differ- 
ent when an anesthesia provider is interrupted in the 
middle of entering a series of medications, our experi- 
ment was not designed to investigate that possibility. 
Fourth, even though faster may not mean better, speed 
of entry along with the number of errors was chosen as 
a reasonable surrogate for ease of use. Fifth, we only 
studied the entry of a drug name, not the dose, even 
though designing of the interface can have a significant 
impact on the drug dosing errors in a health information 
system [8,9]. Sixth, we used an AIMS medication entry 
screen virtually identical to the actual screen in Epic. 
Whether or not our conclusions can directly be extrapo- 
lated to another AIMS - perhaps one based on keyboard 
or mouse-based entry - is unclear, although clearly the 
same testing protocol could be developed and used for 
other systems. 

Conclusion 

Arrangement of data entry elements in a logical fashion 
is an important feature of design of electronic medical 
records. Our results show that the arrangement of drugs 
in a categorical format in the medication order-entry 
screen of an AIMS can result in faster data entry 



compared to an alphabetical arrangement of drugs. 
Results of this quality improvement project were used in 
our departments design of our final intraoperative elec- 
tronic anesthesia record (Figure 4). 

Endnotes 

a The following PubMed search identified 26 articles 
September 20, 2010: ( medical order entry systems 
[MeSH] OR order entr*[TIAB] OR drug list* [TIAB] or 
medication list*[TIAB] OR anesthesia information 
[TIAB] ) AND ( alphabet* [TIAB] OR categor*[TIAB] ) 
AND ( experiment* [TIAB] OR laborator*[TIAB] OR 
participant* [TIAB] OR random* [TIAB] OR simulat* 
[TIAB] OR subject* [TIAB] ). No article was relevant in 
terms of comparing alphabetical versus categorical drug 
lists. When repeated December 18, 2010, there were 31 
articles identified. One is an experimental study evalu- 
ating the impact of interruptions on drug entry errors 
[10]. 
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